feat: wire --pool / topk_retrieve override through brain_ask and run_ablation (#92) - #97

Part 1 for #92 (per #92 (comment)): wires --pool / --topk-retrieve (and BRAIN_TOPK_RETRIEVE) through brain_ask.py and eval/run_ablation.py without changing TOPK_RETRIEVE = 60 default, with before/after tables on both bench/vault (202 notes) and a 3x larger synthetic vault (606 notes) below.

Summary

• src/sqlite_graph_memory/brain_ask.py:
  • Adds _parse_pool(args, env=None) supporting --pool N, --pool=N, --topk-retrieve N, --topk-retrieve=N, and BRAIN_TOPK_RETRIEVE=N (while keeping TOPK_RETRIEVE = 60 untouched as the default).
  • Wires topk=pool into retrieve_candidates(sims, meta, topk=pool) in main().
• eval/run_ablation.py:
  • Adds --pool / --topk-retrieve CLI flag (topk_retrieve parameter on _default_cell, _fill_cell, and parse_grid) to override the default dense pool size for any grid cell that does not explicitly pin topk=....
  • Adds topk-15 (topk: ba.GHOPS) to DEFAULT_GRID alongside topk-20 and topk-80.
• tests/test_brain_ask.py & tests/test_eval.py:
  • Adds test_parse_pool_flag_and_env_override_without_changing_default and test_pool_override_wires_topk_retrieve_without_changing_default.

1. Default TOPK_RETRIEVE = 60 Before vs. After (Zero Regression on gold_202_seed42.jsonl)

Because TOPK_RETRIEVE = 60 remains the default when --pool is omitted, before/after metrics on the frozen gold_202_seed42.jsonl (n=100, --seed 42) are bit-identical (0.000 delta across every class):

2. Measured brain_ask.py --ab --pool 15 Delta (bench/vault)

$ python -m sqlite_graph_memory.brain_ask "Project Heron dependency on Tidewater" --ab --pool 15
A/B RECALL: Project Heron dependency on Tidewater
(Direct=15 vector / Associative=+40 graph; Associative promoted 4 new notes into top-12, 4 of them via graph)

--- DIRECT memory (vector) ---
 1. [  1.68] project-heron
 2. [ -0.96] meeting-2025-12-16-tidewater-sync
 3. [ -1.85] project-redshank
 4. [ -3.89] decision-2025-08-16-heron-deadline
 5. [ -4.37] decision-2025-02-07-heron-deadline
 6. [ -5.15] meeting-2025-08-04-tidewater-sync
 7. [ -5.24] meeting-2025-03-09-tidewater-sync
 8. [ -5.75] meeting-2025-08-25-tidewater-sync
 9. [ -6.10] team-tidewater
10. [ -7.71] concept-backpressure
11. [ -9.07] project-smew
12. [ -9.86] concept-graceful-degradation

--- ASSOCIATIVE memory (vector+graph) ---
 1. [  1.68] project-heron
 2. [ -0.96] meeting-2025-12-16-tidewater-sync
 3. [ -1.85] project-redshank
 4. [ -3.89] decision-2025-08-16-heron-deadline
 5. [ -4.16] person-anselm-asturias  <== +graph
 6. [ -4.37] decision-2025-02-07-heron-deadline
 7. [ -5.06] person-talia-wexley  <== +graph
 8. [ -5.15] meeting-2025-08-04-tidewater-sync
 9. [ -5.24] meeting-2025-03-09-tidewater-sync
10. [ -5.26] person-ilsa-thorsby  <== +graph
11. [ -5.47] person-petra-asturias  <== +graph
12. [ -5.75] meeting-2025-08-25-tidewater-sync

3. --pool Tables Across Two Vaults (--seed 42)

Commands & Reproducibility (Including 606-Note Vault Generator)

# 1. Vault 1: bench/vault (202 notes, 100 gold questions)
BRAIN_INDEX_DIR=/tmp/idx_202 python -m sqlite_graph_memory.index_notes bench/vault
BRAIN_INDEX_DIR=/tmp/idx_202 python eval/build_gold.py bench/vault --n-per-class 30 --seed 42 --out /tmp/gold_202_seed42.jsonl
for pool in 60 20 15; do
  BRAIN_INDEX_DIR=/tmp/idx_202 python eval/run_ablation.py --gold /tmp/gold_202_seed42.jsonl --pool $pool --grid "hops=0;hops=1,ghops=15,gmax=40"
done

# 2. Generate deterministic 3x larger synthetic vault (/tmp/vault_606, 606 notes across alpha/beta/gamma divisions)
python - <<'PY'
import pathlib
src_notes = sorted(pathlib.Path("bench/vault").glob("*.md"))
stems = [p.stem for p in src_notes]
vault2 = pathlib.Path("/tmp/vault_606")
vault2.mkdir(parents=True, exist_ok=True)
for prefix, domain_tag in [("alpha", "Alpha division"), ("beta", "Beta division"), ("gamma", "Gamma division")]:
    for p in src_notes:
        text = p.read_text(encoding="utf-8")
        for s in stems:
            text = text.replace(f"[[{s}]]", f"[[{prefix}-{s}]]").replace(f"[[{s}|", f"[[{prefix}-{s}|")
        lines = text.splitlines()
        out_lines, replaced_h1 = [], False
        for line in lines:
            if not replaced_h1 and line.startswith("# "):
                out_lines.append(f"{line} ({domain_tag})")
                replaced_h1 = True
            else:
                out_lines.append(line)
        (vault2 / f"{prefix}-{p.name}").write_text("\n".join(out_lines) + "\n", encoding="utf-8")
PY

# 3. Vault 2: /tmp/vault_606 (606 notes, 120 gold questions)
BRAIN_INDEX_DIR=/tmp/idx_606 python -m sqlite_graph_memory.index_notes /tmp/vault_606
BRAIN_INDEX_DIR=/tmp/idx_606 python eval/build_gold.py /tmp/vault_606 --n-per-class 30 --seed 42 --out /tmp/gold_606_seed42.jsonl
for pool in 60 20 15; do
  BRAIN_INDEX_DIR=/tmp/idx_606 python eval/run_ablation.py --gold /tmp/gold_606_seed42.jsonl --pool $pool --grid "hops=0;hops=1,ghops=15,gmax=40"
done

Vault 1: bench/vault (202 notes, n=100 gold questions, --seed 42)

--pool 60 (Current Default)

--pool 20

--pool 15 (GHOPS=15)

Vault 2: 3x Synthetic Vault (606 notes, n=120 gold questions, --seed 42)

--pool 60 (Current Default)

--pool 20

--pool 15 (GHOPS=15)

Takeaway

Both vaults show the same pattern:

1. On bridge queries, narrowing --pool from 60 to 20 or 15 raises 1-hop bridge Recall@12 from 0.066 -> 0.141 / 0.179 on Vault 1 (202 notes) and 0.041 -> 0.072 / 0.089 on Vault 2 (606 notes), and lifts overall nDCG@12 from 0.652 -> 0.664 / 0.671 (Vault 1) and 0.500 -> 0.505 / 0.507 (Vault 2).
2. Notice that on Vault 2 (606 notes), vector-only body nDCG@12 improves under --pool 15 (0.706 -> 0.725), while adding 40 1-hop graph neighbors (gmax=40) on top of a 15-note dense pool (graph-pool15 vs vec-pool15) pulls body back down by -0.021 (0.725 -> 0.704, identical to graph-pool60's 0.704). Keeping TOPK_RETRIEVE = 60 as the default while exposing --pool lets callers and ablations tune topk alongside gmax.

Verification

• EVAL_MUTANT=1 pytest -q tests/test_eval.py: 3 failed (red-first ruler intact)
• pytest -q: 268 passed, 2 skipped
• Testament audit: request_id: d581eec6367240079363b14efad053cb (status: passed, 268 passed, 2 skipped)

— TangleClaw'd by Jason Vaughan
Human-directed. Agent-executed. Evidence-backed.
TangleClaw.com